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Abstract — In  future  battlefield  operations,  autonomous 
agents  such  as  Unmanned  Ground  Vehicles  (UGVs)  and 
Unmanned  Airborne  Vehicles  (UAVs)  will  be  projected  to 
the  forefront  for  intelligence,  strike,  search  and  rescue  and 
other  tactical  operations.  The  agents  will  be  organized  in 
clusters  in  order  to  carry  out  such  missions;  different 
clusters  may  execute  different  missions  simultaneously. 
During  the  mission,  the  unmanned  agents  are  supported  by 
sensors  on  the  ground  and  in  the  air,  and  can  receive 
commands  and  send  information  back  to  a  command  ship, 
say.  It  is  clear  that  efficient  communications  between  agents, 
and  from  agents  to  sensors  and  to  command  posts  are 
critical  to  mission  success.  The  goal  of  the  Minuteman 
project  is  to  develop  the  concept  and  initial  prototype  of  an 
agile,  dynamic,  multi-layer  “Internet  in  the  Sky” 
architecture  that  can  deliver  the  “forward  power”  of  the 
unmanned  missions.  The  architecture  consists  of  a  high 
speed,  wireless  Mobile  Backbone  Network  (MBN)  -  with 
point-to-point  wireless  links,  and  local  access  networks 
feeding  to  backbone  nodes.  The  design  is  extremely 
challenging  because  of  the  hostile  environment,  the  need  for 
QoS  support  and  the  unpredictable,  nature  of  the 
requirements.  The  focus  of  this  paper  is  on  scalable 
addressing  and  routing  in  such  a  multiplayer,  mobile 
environment  where  UAVs  can  fly  at  speeds  exceeding 
several  hundreds  miles  per  hour.  We  exploit  the  fact  that 
agents  typically  move  in  groups,  and  achieve  scalability  by 
keeping  track  of  a  “landmark”  for  each  group.  This  is  done 
using  LANMAR,  a  Land-Mark  Ad  hoc  Routing  scheme. 
The  LANMAR  scheme  originally  developed  for  “flat”  ad 
hoc  networks  extends  naturally  to  a  network  with  a  physical 
backbone.  Via  simulation  we  show  that  LANMAR 
maintains  robust,  resilient,  rapidly  restored  connectivity  in 
the  face  of  agent  mobility. 
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1.  Introduction  and  ONR  Program  Overview 

In  future  battlefield  operations,  autonomous  agents  such  as 
Unmanned  Ground  Vehicles  (UGVs)  and  Unmanned 
Airborne  Vehicles  (UAVs)  will  be  projected  to  the  forefront 
for  intelligence,  surveillance,  strike,  enemy  antiaircraft 
suppression,  damage  assessment,  search  and  rescue  and 
other  tactical  operations.  The  agents  will  be  organized  in 
clusters  in  order  to  carry  out  such  missions;  different 
clusters  may  execute  different  missions  simultaneously.  A 
mission  is  generally  assembled  from  various  unmanned 
autonomous  agents  (UGVs,  UAVs,  etc).  The  missions  must 
be  carefully  scheduled,  equipped  with  adequate  resources, 
coordinated  and  monitored  until  completion.  During  the 
mission,  the  unmanned  agents  are  supported  by  sensors  on 
the  ground  and  in  the  air,  and  can  receive  commands  and 
send  information  back  to  a  command  ship,  say.  It  is  clear 
that  throughout  the  various  mission  phases  (from  planning 
to  navigation,  sensor  intelligence  gathering  and  forwarding, 
damage  assessment,  etc.),  efficient  communications 
between  agents,  and  from  agents  to  sensors  and  to  command 
posts  are  critical  to  mission  success 

Addressing  the  above  scenarios  will  be  critical  for  the  Navy. 
In  fact,  future  naval  missions  at  sea  or  shore  will  require 
effective  and  intelligent  utilization  of  real-time  information 
and  sensory  data  to  assess  unpredictable  situations,  identify 
and  track  hostile  targets,  make  rapid  decisions,  and  robustly 
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influence,  control,  and  monitor  various  aspects  of  the  theater 
of  operation.  Littoral  missions  are  expected  to  be  highly 
dynamic  and  extremely  uncertain.  Communication 
interruption  and  delay  are  likely,  and  active  deception  and 
jamming  are  anticipated. 

Efficient  system  solutions  to  the  above  problems  are 
currently  investigated  by  the  Office  of  Naval  Research 
(ONR)  in  a  comprehensive  “Intelligent,  Autonomous 
Networked  Agents”  program.  ONR  envisions  unmanned 
systems  to  have  a  profound  influence  on  future  naval 
operations  allowing  continuous  forward  yet  unobtrusive 
presence  and  the  capability  to  influence  events  ashore  as 
required.  Unmanned  vehicles  have  proven  to  be  valuable  in 
gathering  tactical  intelligence  by  surveillance  of  the 
battlefield.  For  example,  UAVs  such  as  Global  Hawk  are 
rapidly  becoming  integral  part  of  military  surveillance  and 
reconnaissance  operations.  The  goal  is  to  expand  the 
operational  capabilities  of  UAVs  to  include  not  only 
surveillance  and  reconnaissance,  but  strike  and  support 
mission  (e.g.,  command,  control,  and  communications  in  the 
battle  space)  as  well.  This  new  class  of  autonomous 
vehicles  is  foreseen  as  being  intelligent,  collaborative, 
recoverable,  and  highly  maneuverable  in  support  of  future 
naval  operations. 

The  ONR  approach  is  aimed  at  an  integrated  agent-based 
system-of- systems  that  embodies  technologies  that  will 
permit  unmanned  systems  to  move  away  from  platform¬ 
centric  operations  to  network-centric  operations,  while 
exploiting  knowledge  and  power  of  survivable  tiered 
weapons  and  sensors  combined  with  fully  netted  maneuver 
warfare  and  enabling  the  Navy  to  bring  fully  netted  force  to 
the  battle  space.  Netted-Force,  as  shown  in  Figure  1,  is  the 
glue  that  pulls  supporting  technologies  such  as  mission 
planning,  path  planning,  reasoning,  decision  making,  and 
distributed  real-time  computing  and  control  together. 
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Figure  1  -  Netted  Force  through  Distributed  Networks  of 
Intelligent  Agents 


The  development  of  netted-force  hinges  on  three  essential 
technologies: 

(a)  Robust  wireless  connectivity  and  dynamic  networking 
of  autonomous  unmanned  vehicles  and  agents, 

(b)  Intelligent  agents  including:  mobile  codes,  distributed 
databases  and  libraries,  robots,  intelligent  routers, 
control  protocols,  dynamic  services,  semantic  brokers, 
message-passing  entities,  disembodied  code, 

(c)  Decentralized  hierarchical  agent-based  organization. 

As  Figure  1  illustrates,  the  autonomous  agents  have  varying 
domains  of  responsibility  at  different  levels  of  the  hierarchy. 
For  example,  clusters  of  UAVs  operating  at  low  altitude 
(1K-20K  feet)  may  perform  combat  missions  with  a  focus 
on  target  identification,  combat  support,  and  close-in 
weapons  deployment.  Mid-altitude  clusters  (20-50K  feet) 
could  execute  knowledge  acquisition,  for  example, 
surveillance  and  reconnaissance  missions  such  as  detecting 
objects  of  interest,  performing  sensor  fusion/integration, 
coordinating  low-altitude  vehicle  deployments,  and 
medium-range  weapons  support.  The  high  altitude 
cluster(s)  (50K-80K  feet)  provides  the  connectivity.  At  this 
layer,  the  cluster(s)  has  a  wide  view  of  the  theater  and 
would  be  positioned  to  provide  maximum  communications 
coverage  and  will  support  high-bandwidth  robust 
connectivity  to  command  and  control  elements  located  over- 
the-horizon  from  the  littoral/targeted  areas. 

The  hierarchical  agent  organization  has  architectural 
features  useful  for  the  design  of  the  dynamic  network 
architecture.  Higher  levels  of  the  hierarchy  mostly  operate 
over  a  greater  spatial  extent  but  at  slower  time-scales.  The 
reason  is  that  the  transfer  of  data  over  larger  spaces  usually 
requires  more  time,  because  data  transfer  requires  multiple 
hops,  and  in  a  wireless  environment,  the  reliability  of  a  link 
can  degrade  rapidly  with  increasing  range.  Thus,  stronger 
codes  may  be  required  at  the  expense  of  bandwidth.  The 
bandwidth  requirements  could  be  derived  from  the  space- 
time  locus  of  data.  Following  are  some  of  the  essential 
communication  requirements: 

•  Secure  communications  to  deny  information  to  hostile 
forces.  This  is  particularly  challenging  because  the 
envisioned  strength  of  the  autonomous  agents  stems 
from  their  ability  to  share  information  and  perform 
distributed  information  processing  and  fusion; 

•  Low  Probability  of  Detection  and  Interception  and 
Antijamming  capability  in  order  to  penetrate  deep 
into  hostile  territory.  Once  AUVs  are  detected,  hostile 
forces  will  attempt  to  disrupt  the  AUV’s 
communication  system  with  jamming  techniques 
ranging  from  broadband  noise  to  optimum  fraction-of- 
the-band  jammers; 

•  Channel  Capacity:  data  quality,  high  throughput,  and 
high  performance,  for  example,  low  bit  error  rate,  frame 
error  rate,  lost  data,  and  delay; 

•  Dynamic  Network  Resource  Allocation:  reliability, 
redundancy,  availability,  interoperability  of 
communication  links  to  insure  a  high  degree  of 


connectivity,  e.g.,  alternate  transmission  routes  and 
multihop  communications,  in  hostile  environments. 

Functional  flexibility  and  interoperability  of  the  autonomous 
agents  are  essential  to  the  overall  mission  effectiveness,  that 
is,  loss  of  or  malfunction  of  individual  agents  should  only 
result  in  marginal  degradation  of  the  mission.  This  self- 
healing/self-preservation  characteristic  relies  on  the 
autonomy,  which  includes  redundant  functionality, 
adaptation,  and  self-reconfiguration,  as  well  as  robust 
connectivity  of  the  aggregate  system  through:  (a) 
Distribution  and  reallocation  of  essential  functions  amongst 
the  vehicles  in  a  given  cluster;  and  (b)  Transfer  of  agents 
from  one  cluster  to  another. 

These  capabilities  can  only  be  realized  through  adaptable 
dynamic  communication  networks  allowing  reliable,  secure, 
high  throughput  connectivity.  These  networks  can  be 
grouped  as:  (a)  Intra-network  for  secure  communications 
among  the  vehicles  within  the  local  network/line -of- sight; 
(b)  Inter-network  for  secure  communications  between  the 
vehicles  in  adjacent  networks.  Other  significant  and 
challenging  issues  are: 

(a)  Adaptive  Communications.  Agents’  mission  diversity 
and  cooperative  networking  configurations  coupled 
with  the  vehicles’  dynamics  and  mobility  will  demand  a 
communication  infrastructure  that  is  adaptive  and 
dynamic.  The  architecture  must  accommodate 
adjustments  to  changing  channels,  network 
configurations,  data  requirements,  and  security.  The 
focus  is  on  developing  adaptive  connectivity  techniques 
at  various  levels  of  the  hierarchy,  including  the  physical 
layer,  network  layer,  data/information  layer,  and 
security  layer.  In  contrast  to  non-adaptive  schemes  that 
are  designed  relative  to  the  worst-case  channel 
conditions,  adaptive  techniques  take  advantage  of  the 
time- varying  nature  of  wireless  channels.  That  is,  in 
adaptive  techniques  the  goal  is  to  vary  the  transmitted 
power  level,  symbol  rate,  coding  rate/scheme, 
configuration  size,  or  any  combination  of  these 
parameters  in  order  to  improve  the  link  performance 
which  includes  data  rate,  latency,  and  bit  error  rates 
(BER),  while  meeting  the  system  performance 
specifications.  Adaptive  modulation  has  been  shown  to 
increase  the  data  rates  on  flat-fading  channels  by  a 
factor  of  five  or  more.  Additional  coding  can  be  used 
to  obtain  a  reduction  in  transmit  power  or  BER  or  to 
increase  resistance  to  jamming.  Moreover,  the  BER  in 
adaptive  modulation  remains  constant  independent  of 
channel  variations,  which  greatly  improves  reliability  of 
the  wireless  link. 

(b)  Adaptive  QoS.  Adaptive  protocols  that  adjust  to  a 
specific  mission  or  application,  or  can  secure  an 
acceptable  end-to-end  performance  will  be  required. 
Adaptation  may  take  the  form  of  variable-rate  or  multi- 
resolution  compression,  variable-rate  error  correction 


coding,  and  message  prioritization  relative  to  delay 
constraints. 

The  rest  of  the  paper  is  organized  as  follow.  In  section  2  we 
review  the  MINUTEMAN  project.  In  section  3,  we 
introduce  a  scalable  Mobile  Backbone  Network  (MBN) 
infrastructure.  We  address  the  dynamic  Backbone  Node 
(BN)  election  problem  and  introduce  a  new  stable  clustering 
scheme  in  section  4.  In  section  5,  we  present  the  LANMAR 
routing  extension  to  the  MBN.  In  section  6,  our  clustering 
scheme  is  compared  with  other  popular  ones  regarding 
stability  and  our  routing  scheme  is  evaluated  in  a  large-scale 
ad  hoc  network.  Related  work  is  given  in  section  7.  Section 
8  concludes  the  paper. 

2.  The  MINUTEMAN  Project 

The  goal  of  the  Minuteman  (Multimedia  Intelligent  Network 
of  Unattended  Mobile  Agents)  project,  recently  funded  by 
the  Office  of  Naval  Research,  is  to  develop  the  concept  and 
initial  prototype  of  an  agile,  dynamic  “Internet  in  the  Sky” 
architecture  that  can  support  the  demanding 
communications  requirements  of  the  agents  and  can  deliver 
the  “forward  power”  of  the  unmanned  missions.  Here,  we 
briefly  describe  the  challenges  that  such  Internet  in  the  Sky 
design  poses  in  the  face  of  the  unique  requirements  of  the 
unmanned  missions.  We  also  outline  the  innovative 
approaches  that  we  plan  to  undertake  in  order  to  meet  such 
challenges. 

The  first  challenge  is  to  handle  agent  mobility,  which  will 
vary  from  the  roving  speed  of  the  UAGs  all  the  way  to  the 
hundreds  of  miles  per  hour  speed  of  airborne  assets  (UAVs) 
during  an  attack  mission.  The  traditional  Mobile  IP 
approach  will  not  scale  to  large  number  of  mobile  agents, 
high  speeds  and  pervasive  mobility:  the  registration  of  the 
mobile  with  Eoreign  Agents  introduces  excessive  overhead 
and  the  rerouting  via  Home  Agent  and  Eoreign  Agent 
becomes  impractical.  Our  approach  will  be  to  embed 
mobility  support  at  OSI  layer  2,  using  ad  hoc  networking 
and  ad  hoc  routing,  below  IP  (we  will  still  retain,  however, 
the  Mobile  IP  paradigm  for  communications  with  the  wired 
Internet).  Moreover,  we  will  exploit  the  fact  that  agents 
typically  move  in,  and  will  achieve  scalability  by  keeping 
track  of  group  rather  than  individual  movements.  Our 
scalable  approach  to  group  mobility  management  and 
routing  has  been  implemented  in  the  LANMAR  (Landmark 
ad  hoc  routing)  architecture  [18].  Eor  the  MBN  network  the 
LANMAR  architecture  has  been  extended  to  large  node 
populations  and  large  geographical  distances  using  multi¬ 
layering  (ie,  backbone)  concepts.  In  the  paper,  we  will 
present  initial  LANMAR  results  for  representative 
scenarios.  We  will  evaluate  the  ability  of  LANMAR  to 
maintain  robust,  resilient,  rapidly  restored,  nearly  optimal 
(in  terms  of  path  length)  connectivity  in  the  face  of  agent 
mobility. 


The  robust,  all-time  connectivity  provided  by  LANMAR  is 
critical,  but  it  is  not  sufficient  to  carry  out  successful 
missions.  The  UAVs  gathering  intelligence  at  the  forefront 
must  be  able  to  transmit  multimedia  (eg,  compressed  video) 
streams  with  bandwidth  guarantees  across  the  backbone 
network  to  other  clusters  of  mobile  agents  preparing  the 
attack;  or,  to  the  commander  on  the  ship.  Thus,  a  second 
important  challenge  for  the  airborne  Internet  is  to  support 
Quality  of  Service  in  terms  of  bandwidth,  response  time 
delay,  and  delay  variation.  We  are  planning  a  multi-layer 
approach  to  QoS  that  will  include  backbone  beam  forming 
at  the  radio  layer,  MAC  layer  scheduling,  network  layer 
QoS  routing.  Call  Acceptance  Control  and  backbone  path 
pinning  by  means  of  label  switched  paths  and  MPLS  (Multi 
Protocol  Label  Switching).  MPLS  will  provide  the 
flexibility  to  forward  individual  and/or  aggregated  flows  on 
QoS  compliant  multiple  paths  selected  by  the  QoS  routing 
algorithm,  overcoming  the  limitations  of  traditional  shortest 
path  routing.  These  concepts  are  implemented  in  the 
“Mobile  Backbone  Network”  architecture  that  builds  upon 
the  LANMAR  connectivity  management  and  provides  QoS 
where  needed.  QoS  support  requires  the  allocation  and 
“alignment”  of  several  network  resources  (eg,  backbone 
UAVs  in  strategic  positions).  If  the  UAVs  are  destroyed  or 
reassigned  to  a  more  critical  mission,  QoS  will  be  gracefully 
degraded,  possibly  all  the  back  to  basic  LANMAR 
connectivity. 

A  third  critical  requirement  of  our  architecture  is  to 
dynamically  adjust  to  environment  changes  that  are  either 
due  to  natural  causes  (eg,  radio  propagation  irregularities, 
fading,  mobility,  obstacles,  battery  power  depletion,  etc)  or 
to  enemy  actions  (eg,  UAV  destruction,  radio  jamming,  etc). 
In  view  of  such  abrupt  and  often  unpredictable  changes, 
network  protocols  and  applications  must  react  in  concert 
and  must  adaptively  readjust  to  the  new  situation.  We  will 
discuss  various  adaptive  protocol  features  (both  intra  and 
interlayer)  that  are  being  designed  in  our  architecture, 
specifically  to  address  these  changes.  Moreover,  the  total 
unpredictability  of  these  changes  makes  it  impossible  to 
provide  “guaranteed”  QoS,  as  it  is  generally  done  in 
commercial  networks.  Instead,  the  concept  of  guaranteed 
QoS  is  replaced  by  that  of  “adaptively  renegotiable”  QoS.  In 
the  paper  we  will  elaborate  on  a  particularly  important 
example  of  adaptation  in  the  battlefield,  namely,  the 
adjustment  of  compressed  video  (say,  MPEG  4)  parameters 
in  order  to  make  best  use  of  the  existing  network  resources. 

Dynamic  adaptation  is  also  required  in  the  assembly  of 
resources  to  launch  a  mission  and  to  track  its  progress.  In 
this  respect,  the  unique  feature  of  the  unmanned  agent 
system  is  that  some  agents  can  support  multiple  functions. 
For  example,  a  UAV  can  be  used  for  communications,  as  a 
node  of  the  Mobile  Backbone  Network;  as  well  as  for 
intelligence,  to  gather  video  and  images  as  part  of  a  scouting 
mission.  Thus,  planning  a  mission  requires  the  allocation  of 
limited  resources  and  possibly  the  “reallocation”  of 
resources  from  background  missions  to  top  priority 
missions.  Monitoring  and  dynamic  reallocation  of  resources 


based  on  time  changing  priorities,  along  with  QoS 
renegotiation,  is  performed  by  a  distributed,  systems  wide 
Adaptive  Resource  Monitoring  and  Management  Network 
(ARMMNET).  ARMMNET  will  permit  to  dynamically 
reallocate  across  multiple  simultaneous  missions  the  various 
battlefield  resources  (from  communications  to  CPU  power, 
memory  and  databases)  in  the  most  efficient  manner. 

Advanced  applications  such  as  Automatic  Target 
Recognition  (ATR)  require  the  gathering  of  video  and 
sensor  information  from  vast  areas  in  the  battlefield  in  order 
to  determine  presence  and  type  of  targets.  This  information 
must  be  received  with  extremely  tight  accuracy  and  time 
constraints  in  order  to  execute  a  successful  strike  mission, 
say.  Brute  force  scanning  of  the  entire  area  may  not  be 
feasible  -  it  may  require  too  much  time  given  the  available 
UAV  and  sensor  assets.  In  order  to  accomplish  the  goal 
within  the  required  constraints,  we  propose  to  maintain  a 
distributed  Information  Database  that  can  provides  global 
information  about  assets  in  the  battlefield  as  well  as  video 
and  images  captured  during  routine  surveillance.  The 
Information  Database  will  permit  to  “guide”  UAV  clusters 
in  the  search  to  the  critical  areas  and  to  supplement  the 
UAV  and  sensor  image  data  with  stored  information,  thus 
reducing  the  time  to  target  detection  and  recognition.  The 
maintenance  of  a  timely  and  accurate  Information  Database 
in  our  environment  poses  several  new  challenges,  including: 
distributed,  fault  tolerant  implementation;  ability  to  answer 
queries  with  variable  degree  of  accuracy  depending  on  time 
constraints;  and,  careful  tradeoff  between  the  background 
refresh  rate  (and  thus  accuracy)  and  the  use  of  limited 
communications  and  sensor  resources. 

The  dynamic  adaptation  to  the  unpredictable,  hostile 
environment  requires  the  support  of  advanced, 
programmable  radios  and  of  adaptive  modulation  and 
channel  encoding  schemes.  The  goal  here  is  to  achieve  the 
best  use  of  the  available  spectrum  while  providing  the  radio 
range,  beam  directivity  and  channel  quality  required  by  the 
upper  protocol  layers.  An  important  contribution  of  this 
project  will  be  the  development  of  “modular”  radios  that 
utilize  advanced  MIMO  and  OFDM  techniques  and  can  be 
dynamically  reconfigured  to  fit  the  needs  of  a  low  power 
stationary  sensor  as  well  as  the  challenging  demands  of  a 
fast  flying  UAV  with  video  capture. 

Finally,  the  demonstration  of  our  highly  adaptive  suite  of 
protocols  will  itself  be  a  challenge.  It  will  not  suffice  to 
demonstrate  each  component  in  isolation:  the  key  is  the 
successful  interoperation  of  the  components  and  the 
cooperative,  interlayer  adaptation  to  unpredictable  changes 
in  the  environment.  To  this  end,  we  will  develop  a  novel, 
“hybrid”  simulator  capability  that  will  allow  to  interface 
“real”  applications  to  simulated  innercore  network  protocols 
for  a  widely  ranging  set  of  configuration  parameters 
(number  of  nodes,  speeds,  etc).  The  hybrid  simulation 
testbed  will  be  an  essential  complement  of  the  hardware 
testbed  which  is  by  practical  necessity  limited  in  number, 
speed  and  geographic  scope. 


In  summary,  the  adaptive,  unmanned  agent  “Internet  in  the 
Sky”  project  will  require  an  unprecedented  degree  of 
adaptivity  in  the  design  of  the  various  protocol  layers,  from 
radio  to  applications,  and  the  development  of  new  adaptive 
middleware  (ARMMNET)  and  new  hybrid  simulation 
techniques  for  testbed  deployment  and  evaluation.  As  the 
trend  in  modern  communications  systems,  both  military  and 
civilian,  is  to  become  increasingly  more  complex, 
autonomous  and  “adaptive”,  we  believe  that  our  unique, 
innovative  solutions  can  be  effectively  transferred  in  the 
future  to  several  other  application  domains. 

The  project  has  begone  in  Dec  2000.  Progress  has  been 
reported  in  all  the  above  mentioned  tasks.  In  this  paper  we 
focus  on  scalable  routing  in  the  MINUTEMAN  multilevel 
network  architecture  consisting  of  a  high  speed  mobile 
backbone  and  of  local  access  subnetworks  feeding  to  each 
of  the  backbone  nodes.  A  companion  paper  in  this  session 
describes  the  Backbone  Network  design  philosophy.  Our 
paper  introduces  a  very  flexible,  scalable  routing  solution 
that  handles  mobility  and  can  work  on  top  of  any  arbitrary 
network  infrastructure  (eg,  physical  backbone  network)  and 
any  “local  scope”  routing  scheme.  We  exploit  the  fact  that 
agents  typically  move  in  groups,  and  achieve  scalability  by 
keeping  track  of  a  “landmark”  for  each  group.  This  is  done 
using  LANMAR,  a  Land-Mark  Ad  hoc  Routing  scheme. 
The  LANMAR  scheme  was  originally  developed  for  “flat” 
ad  hoc  networks;  but,  it  extends  naturally  to  a  network  with 
a  physical  backbone. 

3.  Ad  Hoc  Scalable  Routing 

The  ad  hoc  wireless  networking  technology  shows  great 
potential  and  importance  in  many  situations  because  of  its 
independence  of  a  fixed  infrastructure  and  its  instant 
deployment  and  easy  reconfiguration  capabilities.  Usually,  a 
mobile  ad  hoc  network  (MANET)  is  assumed  to  be 
homogeneous.  However,  a  flat  ad  hoc  network  has  poor 
scalability[l][2][ll].  In  [1],  theoretical  analysis  implies  that 
even  under  the  optimal  circumstances,  the  throughput  for 
each  node  declines  rapidly  toward  zero  while  the  number  of 
nodes  is  increased.  This  is  proved  in  an  experimental  study 
of  scaling  laws  in  ad  hoc  networks  employing  IEEE  802.11 
radios  presented  in  [2].  The  measured  per  node  throughput 
declines  much  faster  in  the  real  testbed  than  in  theory. 
Simulation  results  in  [10]  also  demonstrated  that  while 
routing  protocols  are  applied,  their  control  overhead  would 
consume  most  available  bandwidth  when  the  traffic  is 
heavy.  Besides  limitation  of  available  bandwidth,  the  “many 
hop”  paths  in  large-scale  network  are  prone  to  break  and 
cause  many  packet  drops.  Packet  drop  can  be  treated  as 
waste  of  bandwidth  and  worsen  network  performance.  All 
these  issues  prevent  the  flat  ad  hoc  network  from  scaling  to 
large-scale.  Thus,  a  new  methodology  is  needed  for  building 
a  large-scale  ad  hoc  network.  An  emerging  promising 
solution  is  to  build  a  physically  hierarchical  ad  hoc  network 
and  mobile  wireless  backbones. 


Our  proposed  hierarchical  ad  hoc  network  structure  is  called 
an  ad  hoc  network  with  mobile  backbones  (MBN).  A 
general  picture  of  a  two  level  MBN  is  demonstrated  in 
Eigure  2.  Among  the  mobile  nodes,  some  nodes,  named 
backbone  nodes  (BNs),  have  an  additional  powerful  radio  to 
establish  wireless  links  among  themselves.  Thus,  they  form 
a  higher-level  network  called  a  backbone  network.  Since  the 
backbone  nodes  are  also  moving  and  join  or  leave  the 
backbone  network  dynamically,  the  backbone  network  is 
exactly  an  ad  hoc  network  running  in  a  different  radio  level. 
Multilevel  MB  Ns  can  be  formed  recursively  in  the  same 
way. 


Backbone  Network 


Three  critical  issues  are  involved  in  building  such  a  MBN: 
(1)  the  optimal  number  of  BNs;  (2)  BN  deployment,  and; 
(3)  routing.  Assuming  that  the  number  of  BNs  has  been 
determined,  the  second  important  issue  is  how  to  deploy 
them  in  the  field.  The  main  challenges  in  carrying  out  an 
efficient  deployment  are  mobility  and  BN  failures.  Using  a 
clustering  scheme  to  elect  the  BNs  is  a  natural  choice  since 
clustering  has  been  widely  used  in  the  past  to  partition 
nodes  into  small  sets  and  to  form  hierarchical  networks 
[6]  [7].  However,  a  major  drawback  of  current  clustering 
schemes  is  cluster  instability  in  the  face  of  mobility  [6]. 
Unstable  clusters  lead  to  frequent  cluster  head  changes  and 
thus  backbone  node  changes.  The  backbone  topology  would 
then  be  too  dynamic  to  be  tracked  by  routing  and  too 
unpredictable  to  be  relied  upon  for  QoS  support.  In  the 
sequel,  we  will  present  a  new  fully  distributed  clustering 
scheme  that  achieves  good  stability. 

Routing  also  critically  affects  the  hierarchical  network 
performance.  Simply  stated,  routing  must  utilize  the 
wireless  backbone  links  efficiently.  The  main  challenge  that 
sets  wireless  networks  apart  from  the  wired  Internet  is 
mobility:  in  an  Internet  like  routing  scheme  address  prefixes 
would  need  to  be  continuously  changed  as  nodes  move!  The 
overhead  associated  with  address  management  would  easily 
offset  the  routing  control  traffic  and  routing  table  size 
reductions  offered  by  the  hierarchical  structure.  Landmark 
Ad  Hoc  Routing  (LANMAR)  has  proven  to  be  a  very 
effective  scheme  in  large  networks  with  group  mobility 
[17]  [18].  In  this  paper,  we  extend  LANMAR  to  the  MBN 
architecture.  The  extended  version  retains  the  simplicity  of 


the  traditional  “flat”  scheme.  Yet,  it  preserves  all  the  typical 
backbone  strategy  benefits,  namely,  short  paths  to  remote 
nodes,  low  end-to-end  delay,  high  quality  links,  augmented 
network  capacity,  and  enhanced  QoS  support.  Moreover, 
LANMAR  exploits  the  hierarchical  structure  by  reducing 
control  overhead  and  propagating  routing  information  more 
promptly. 

4.  Backbone  Node  Deployment  and 
Clustering 

One  way  to  deploy  the  backbone  network  is  to  pre-compute 
first  the  optimal  number  of  backbone  nodes  (BNs)  required 
by  the  given  initial  node  layout.  Then,  one  distributes  the 
BNs  uniformly  in  the  field  at  initialization.  However,  this  2- 
step  procedure  has  two  problems.  Eirst,  the  BNs  move,  thus 
after  a  while,  some  BNs  may  collide  or  anyway  interfere 
with  each  other;  while  some  areas  may  be  uncovered. 
Secondly,  BNs  may  fail  or  even  be  destroyed.  New  BNs 
must  be  deployed  to  replace  the  failed  ones.  A  static,  a  priori 
allocation  and  deployment  cannot  efficiently  fulfill  both 
requirements. 

Our  proposed  solution  is  to  combine  allocation  (of  number 
of  BNs)  and  deployment  by  initially  assigning  redundant 
backbone  capable  nodes  and  letting  the  election  procedure 
choose  the  active  backbone  set  dynamically  to  meet  the 
changing  requirements.  A  node  is  backbone  capable  if  it  has 
the  physical  radio  capacity  to  communicate  with  other 
backbone  nodes  and  join  the  backbone  network.  If  the 
backbone  capable  nodes  are  redundant,  ie,  are  in  more 
ample  supply  than  strictly  needed,  only  a  subset  of  them 
joins  the  backbone  at  any  given  time.  The  remaining 
candidates  are  kept  as  spare  nodes.  When  one  BN  is 
destroyed  or  moves  out  of  a  certain  area,  a  new  BN  will  be 
dynamically  selected  from  the  backbone  capable  set  to 
replace  it.  If  two  backbone  nodes  move  too  close  to  each 
other,  one  of  them  will  give  up  its  backbone  role. 

The  procedure  to  select  backbone  nodes  from  capable  nodes 
is  called  backbone  election.  It  should  be  dynamically 
performed.  It  should  lead  to  a  proper  number  of  backbone 
nodes  uniformly  covering  the  entire  area.  Clustering  has 
been  traditionally  used  to  select  subset  of  nodes.  In  fact,  it 
was  also  proposed  to  form  “logical”  ad  hoc  network 
hierarchies  [6]  [7].  Here,  we  will  use  it  to  create  a  “physical” 
hierarchy.  In  the  sequel,  we  briefly  review  some  options  and 
then  introduce  our  solution. 

Random  Competition  based  Clustering  (RCC) 

Many  clustering  schemes  have  been  proposed  in  the 
literature  [3] [4] [5] [6].  Among  them,  the  Lowest  ID  (LID) 
and  Highest  Degree  (HD)  algorithms  are  widely  used  due  to 
their  simplicity.  The  detail  of  the  two  algorithms  can  be 
found  in  [3]  [4].  Previous  research  in  clustering  mainly 
focuses  on  how  to  form  clusters  with  a  good  geographic 
distribution,  such  as  minimum  cluster  overlap,  etc. 


However,  stability  is  also  an  important  criterion,  especially 
when  clustering  is  used  to  support  routing.  In  particular,  in 
our  hierarchical  structure,  stability  of  backbone  nodes  is  a 
must.  Previous  clustering  schemes  cannot  meet  such  a 
requirement. 

Targeting  both  stability  and  simplicity,  we  have  designed  a 
new  scheme  called  Random  Competition  Clustering  (RCC). 
The  main  idea  is  that  any  candidate  node,  which  currently 
does  not  belong  to  any  cluster,  can  initiate  a  cluster 
formation  by  broadcasting  a  packet  to  claim  itself  as  a 
cluster  head.  The  first  node,  which  broadcast  such  a  packet, 
will  be  elected  as  the  cluster  head  by  its  neighbors.  All  the 
immediate  neighbors,  after  hearing  this  broadcast,  give  up 
their  right  to  be  a  cluster  head  and  become  members  of  the 
cluster.  Cluster  heads  have  to  periodically  broadcast  a 
cluster  head  claim  packet  to  maintain  their  status.  Since 
there  is  a  delay  from  when  one  node  broadcast  its  cluster 
head  claim  packet  to  when  this  packet  is  heard  by  its 
neighbors,  several  neighbor  nodes  may  broadcast  during  this 
period.  To  reduce  such  concurrent  broadcasts,  we  introduce 
a  random  timer.  Each  node  defers  a  random  time  before  its 
cluster  head  claim.  If  it  hears  a  cluster  head  claim  during 
this  random  time,  it  then  gives  up  its  broadcast.  The  idea  of 
“first  claim  node  wins”  (independently  of  ID  number  or 
connectivity  degree)  was  first  proposed  in  the  Passive 
Clustering  scheme  in  [8].  The  Eirst  Claim  Wins  scheme 
favors  the  Cluster  Head,  which  can  be  challenged  only  by 
preexisting  Cluster  Heads.  In  Passive  Clustering,  clusters 
are  formed  on  demand,  when  user  traffic  is  present.  In 
absence  of  traffic,  the  clusters  are  dissolved.  Our  scheme  is 
active  clustering  (as  the  election  is  carried  out  continuously 
in  the  background);  but  we  nevertheless  use  the  same 
concepts  of  “first  declaration”  and  explicit  random  timer.  Of 
course,  the  random  timer  cannot  completely  solve  the 
concurrent  broadcast  problem.  When  the  concurrent 
broadcasts  happen,  we  use  the  node  ID  to  solve  the  conflict. 
The  node  with  lower  ID  will  become  the  cluster  head. 

Our  Random  Competition  based  Clustering  (RCC)  scheme 
is  more  stable  than  traditional  clustering  schemes  such  as 
LID  and  HD.  In  the  LID  scheme,  when  the  cluster  head 
hears  a  node  with  a  lower  ID,  it  will  immediately  give  up  its 
cluster  head  role.  Similarly,  in  the  HD  scheme,  when  a  node 
acquires  more  neighbors,  the  cluster  will  also  be 
reconfigured. 

Due  to  node  mobility,  such  events  happen  very  frequently. 
In  RCC,  one  node  only  gives  up  its  cluster  head  position 
when  another  cluster  head  moves  near  to  it.  Since  cluster 
heads  are  usually  at  least  two  hops  away,  clusters  formed  by 
RCC  are  much  more  stable. 

The  low  control  overhead  of  our  scheme  is  clear.  In  the 
lowest  ID  and  highest  degree  clustering  schemes,  each  node 
has  to  know  the  complete  information  of  neighbor  nodes.  In 
our  scheme,  only  the  cluster  heads  need  to  broadcast  a  small 
control  packet  periodically.  All  other  nodes  just  keep  silent. 


Multihop  Clustering 

Usually  the  clustering  schemes  are  one  hop  based,  that  is  the 
cluster  head  can  reach  all  members  in  one  hop.  This  is  not 
suitable  for  backbone  node  election.  We  want  to  control 
(and  in  fact  optimize)  the  number  of  elected  BNs.  To 
achieve  this,  we  extend  our  clustering  scheme  to  form  K- 
hop  clusters.  Here,  K-hop  means  that  a  cluster  head  can 
reach  any  one  of  its  members  in  at  most  K  hops.  By 
adjusting  the  parameter  K,  we  can  approximately  control  the 
number  of  cluster  heads.  Bigger  K  means  fewer  cluster 
heads,  thus  fewer  BNs. 

In  K-hop  clustering,  each  node  forwards  the  cluster  head 
claim  packet  received  from  its  cluster  head.  A  mobile  node 
will  select  the  nearest  cluster  head  within  its  K-hop  scope  to 
be  its  cluster  head.  If  there  is  no  cluster  head  within  a  K-hop 
scope,  a  node  claims  itself  as  a  cluster  head  after  deferring 
for  a  random  time.  In  K-hop  clustering,  the  probability  of 
concurrent  cluster  head  claims  is  relatively  high  due  to  the 
longer  time  for  propagating  cluster  head  claim  packets  K- 
hops  away.  The  random  time  delay  plays  a  very  important 
role  here. 

5.  Scalable  Routing  Scheme 

After  the  BNs  are  elected,  powerful  backbone  radios  are 
used  to  connect  BNs  and  form  a  backbone  network.  Now, 
the  critical  issue  is  routing.  The  backbone  links  among  BNs 
provide  “short  cuts”  and  high  bandwidth.  Routing  must  be 
able  to  exploit  backbone  links  for  remote  destinations. 
Moreover,  since  BNs  may  fail  or  even  be  destroyed,  routing 
must  be  reliable  and  tolerant  of  such  failures.  In  this  section, 
we  introduce  Landmark  Ad  Hoc  Routing  (LANMAR) 
[17]  [18]  and  propose  to  extend  it  to  include  also  the  MBN, 
yet  preserving  its  scalability  and  fault  tolerant  properties. 

Landmark  Ad  Hoc  Routing  (LANMAR) 

LANMAR  is  a  scalable  routing  protocol  for  large,  mobile, 
“flat”  ad  hoc  wireless  networks  [17]  [18].  It  assumes  that  the 
network  is  grouped  into  logical  subnets  in  which  the 
members  have  a  commonality  of  interests  and  are  likely  to 
move  as  a  “group”  (e.g.,  a  team  of  co-workers  at  a 
convention;  or  tanks  in  a  battalion,  or  UAVs  in  an 
unmanned  scouting  mission).  The  existence  of  such  logical 
groups  can  be  efficiently  reflected  in  the  addressing  scheme. 
We  assume  that  a  two  level,  IP  like  MANET  (Mobile  Ad 
hoc  NET)  address  is  used  consisting  of  a  group  ID  (or 
subnet  ID)  and  a  host  ID,  i.e.  <Group  ID,  Host  ID>.  The 
group  ID  may  change  from  time  to  time  as  a  node  is 
reassigned  to  a  different  group  (e.g.  task  force  in  a  military 
scenario).  The  Host  ID  is  fixed  and  typically  corresponds  to 
the  hardwired  device  address.  Such  MANET  address 
uniquely  identifies  each  node  in  the  network  (though  it  is 
possible  for  a  node  to  belong  simultaneously  to  more  then 


one  group  and  thus  has  more  than  one  address).  Similar  to 
an  IP  network,  the  packet  is  routed  to  the  group  first,  and 
then  to  the  Host  within  the  group.  The  challenge  is  “find” 
the  group  in  a  large,  mobile  network. 

LANMAR  uses  the  notion  of  landmarks  to  keep  track  of 
such  logical  groups.  Each  logical  group  has  one  node 
serving  as  “landmark”.  The  landmark  advertises  the  route  to 
itself  by  propagating  a  Distance  Vector,  e.g.  DSDV 
(Destination  Sequences  Distance  Vector)  [15].  Eurther,  the 
LANMAR  routing  scheme  is  always  combined  with  a  local 
routing  algorithm,  e.g.  Fisheye  State  Routing  (FSR)  [20]. 
FSR  is  a  link  state  routing  algorithm  with  limited  “scope” 
feature  for  local,  low  overdead  operation.  Namely,  FSR 
knows  the  routes  to  all  nodes  within  a  predefined  Fisheye 
scope  (eg,  3  hops)  from  the  source.  For  nodes  outside  of  the 
Fisheye  scope,  the  landmark  distance  vector  must  be 
inspected  for  directions.  As  a  result,  each  node  has  detailed 
topology  information  about  nodes  within  its  Fisheye  scope 
and  knows  distance  and  routing  vector  (ie,  direction)  to  all 
landmarks. 

When  a  node  needs  to  relay  a  packet  to  a  destination  is 
within  its  Fisheye  scope,  it  obtains  accurate  routing 
information  from  the  Fisheye  Routing  Tables.  The  packet 
will  be  forwarded  directly.  Otherwise,  the  packet  will  be 
routed  towards  the  landmark  corresponding  to  the 
destination  logical  subnet,  which  is  read  from  the  logical 
address  field  in  the  MANET  address.  Thus,  when  the 
packet  arrives  within  the  scope  of  the  destination,  it  may  be 
routed  to  it  directly  without  ever  going  through  the 
landmark. 

LANMAR  in  the  Mobile  Backbone  Network 

In  the  original  LANMAR  scheme,  we  route  the  packet 
toward  the  corresponding  remote  landmark  along  the 
(typically  long)  multi-hop  path  advertised  by  the  Distance 
Vector  algorithm.  If  there  is  an  MBN,  the  min  hop  path  will 
generally  include  some  of  the  Backbone  links.  Thus,  in 
practice  we  route  the  packet  to  the  nearest  BN.  This  local 
BN  then  forwards  the  packet  to  a  remote  BN  near  the 
destination  landmark  via  the  backbone.  Finally,  the  remote 
BN  delivers  the  packet  to  the  remote  landmark  or  directly  to 
destination  if  it  is  within  its  Fisheye  scope.  This  will  greatly 
reduce  the  number  of  hops.  This  procedure  is  illustrated  in 
Figure  3.  We  can  see  that  by  utilizing  the  backbone  links, 
the  6  hop  path  is  reduced  to  be  3  hops  long,  a  great 
improvement!  Note  that  the  routing  within  the  MBN  need 
not  be  DSDV.  In  fact,  in  the  Minutemen  project,  MBN 
routing  is  a  form  of  QoS  as  described  in  a  companion  paper 
in  this  session.  The  Landmarks  are  mapped  to  BNs  (multiple 
mappings  are  possible  for  fault  tolerance).  The  route  within 
the  MBN  is  computed  by  the  MBN  unique  routing 
algorithm  and  may,  in  fact,  satisfy  given  QoS  constraints. 


LANMAR  routing  in  ad  hoc  network  without  backbones 


LANMAR  routing  in  ad  hoc  network  with  backbones 


Figure  3  -  LANMAR  routing  in  MEN 

A  possible  implementation  of  the  MBN/LANMAR  scheme 
is  as  follows.  First,  all  mobile  nodes,  including  BNs,  are 
running  the  original  LANMAR  routing  on  the  “local”  links 
via  the  short-range  radios.  This  is  the  foundation  for  falling 
back  to  “flat”  multi-hop  routing  if  the  MEN  fails.  Secondly, 
a  BN  will  periodically  broadcast  its  landmark  map  (ie,  the 
landmark  distance  vector)  to  neighbor  BNs  via  the  backbone 
links.  The  neighbor  BNs  will  treat  this  packet  as  a  normal 
landmark  update  packet.  Since  this  higher  level  path  is 
usually  shorter,  it  will  replace  the  long  multi-hop  paths. 
From  landmark  updates  the  ordinary  nodes  thus  learn  the 
best  path  to  the  remote  landmarks,  including  the  paths  that 
utilize  the  backbone  links.  Each  BN  needs  to  record  the 
radio  interface  to  the  next  hop  on  each  advertised  path  in 
order  to  route  packets  through  the  correct  radios  later.  As 
discussed  earlier,  the  routing  within  the  MEN  need  not  be 
“shortest  path”-  it  may  in  fact  be  QoS  routing. 

One  important  feature  of  the  proposed  routing  scheme  is 
reliability  and  fault  tolerance.  The  ordinary  nodes  are 
prevented  from  knowing  the  backbone  links  explicitly.  The 
backbone  links  are  automatically  learned  via  routing 
broadcasts  from  BNs.  Now,  suppose  a  BN  of  one  group  is 
destroyed  by  enemies,  the  shorter  paths  via  this  BN  will 
soon  expire.  Then  new  landmark  information  broadcasted 
from  other  nodes  will  replace  the  expired  information.  Thus, 
in  the  worst  case,  routing  in  this  group  goes  back  to  original 
landmark  routing  while  other  groups  with  BN  support  can 
still  benefit  from  the  backbone  short  cuts.  When  all 
backbone  capable  nodes  are  disabled,  the  whole  network 
becomes  a  “flat”  ad  hoc  network  running  the  original 
LANMAR  routing,  which  can  still  provide  connectivity,  yet 
at  lower  performance  (longer  paths;  no  QoS  support). 

In  this  paper  we  assumed  a  very  simple  DSDV  routing 
scheme  in  the  MBN,  with  omnidirectional  antennas, 
neighbor  discovery,  and  distance  vector  routing  support  to 
landmarks.  This  scheme  is  sufficient  to  provide  “short  cut” 
benefits  across  the  backbone.  More  elaborate  and  efficient 
MBN  configurations  (e.g.  point  to  point  links)  and  routing 
schemes  (e.g.  Link  State)  are  currently  being  investigated  in 
the  MINUTEMAN  project,  as  described  by  a  companion 
paper  by  Rubin  et  al.  With  MBN  Link  State  routing,  each 
BN  can  advertise  the  landmarks  within  its  reach.  A  BN  can 


then  select  the  most  cost  effective  route  within  the  backbone 
to  the  intended  destination.  This  makes  QoS  support 
possible. 

LANDMAR  AND  IP  ROUTING 

LANMAR  is  a  MANET  routing  protocol.  As  such,  it  only 
supports  routes  inside  the  ad  hoc  network.  In  order  to  route 
packets  to/from  the  Global  Internet,  other  mechanisms  are 
required.  In  particular,  each  node  is  assigned  an  IP  address 
(IPv4  or  IPv6).  This  IP  address  is  used  by  Corresponding 
nodes  in  the  Global  Internet  to  reach  our  mobile  node  using 
Mobile  IP  via  the  Home  Agent  and  the  Eoreign  Agent  (PA). 
The  PA  is  a  Name  Server  in  the  ad  hoc  network,  that 
provides  the  mapping  from  IP  address  to  LANMAR 
address.  Mobile  nodes  refresh  the  Name  Server  data  base 
periodically  as  well  as  when  they  join  a  new  group.  In  IPv6 
the  local  address  field  becomes  the  MANET  address, 
namely  the  LANMAR  address.  This  helps  remove  the 
inefficiencies  of  Mobile  IPv4  (tunneling  and  triangular 
routing).  More  details  on  this  topic  are  found  in  the  paper  by 
P.Templin  at  al,  also  in  this  session. 


6.  Simulation  Experiments 

In  this  section,  we  evaluate  the  efficiency  of  the  clustering 
and  routing  solutions  so  far  proposed.  We  use 
GlomoSim/Qualnet  [16],  a  packet  level  network  simulation 
platform  for  ad  hoc  networks  based  on  the  parallel  language 
PARSEC.  We  begin  with  the  Random  Competition  based 
Clustering  (RCC)  algorithm.  We  compare  the  stability  of 
our  algorithm  with  the  Lowest  ID  (LID)  and  Highest  Degree 
(HD)  algorithms.  Since  we  are  targeting  large  scale 
networks,  we  deploy  1000  mobile  nodes  in  a  “terrain”  of 
size  3200mX3200m.  Each  mobile  node  has  an  IEEE  802.11 
wireless  radio  with  transmission  range  175m.  The  DCP 
mode  of  IEEE  802.11  is  used  and  channel  bandwidth  is  set 
to  2Mbps.  The  node  mobility  model  is  random  waypoint 
mobility  [14].  In  our  simulation,  the  pause  time  is  kept  as  30 
seconds  and  we  vary  the  mobility  speed  to  observe  the 
stability  of  clusters.  Simulation  time  of  each  run  is  6 
minutes. 

The  stability  of  clusters  includes  two  parts,  the  stability  of 
the  cluster  head  and  the  stability  of  the  cluster  members. 
These  are  measured  by  two  different  metrics:  average 
lifetime  of  a  cluster  head  and  average  membership  time  of  a 
cluster  member.  In  the  MBN,  average  lifetime  of  a  cluster 
head  is  exactly  the  average  lifetime  of  a  backbone  node.  In 
our  simulation,  we  only  implement  the  basic  clustering 
scheme  without  considering  the  “gateway”  node  selection  as 
in  [3]  [4]  etc. 

Cluster  Stability 

Usually,  clustering  is  performed  to  form  one  hop  clusters. 
Thus,  here  we  compare  the  stability  of  one-hop  clusters. 
Again  one  hop  means  that  the  cluster  head  can  reach  all  its 


members  in  one  hop.  The  simulation  results  are  given  in 
Figure  4  and  Figure  5. 


Figure  4  -  Average  lieftime  of  cluster  head  under  mobility 


Figure  5  -  Average  membership  time  under  mobility 


From  Figure  4  and  Figure  5,  we  can  see  our  clustering 
algorithm  is  more  stable  than  the  lowest  ID  and  highest 
degree  algorithms  in  both  low  mobility  and  high  mobility 
scenarios. 

Figure  6  and  Figure  7  show  cluster  head  and  membership 
stability  under  the  RCC  algorithm  when  the  hop  scope  of 
the  cluster  (ie,  K-hop  value)  increases  from  1  to  4.  As 
intuitively  expected,  when  the  number  of  hops  increases,  the 
cluster  becomes  larger  and  thus  more  stable  under  the  “first 
declaration  wins”  rule. 
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Figure  7  -  Average  membership  time  of  multihop  clustering 
Routing  Algorithm  Performance 

In  this  section,  we  compare  the  LANMAR  extension  in 
MEN  with  the  original  LANMAR  routing  and  AODV[13], 
a  popular  on-demand  routing  protocol,  in  the  flat  ad  hoc 
network.  The  basic  environment  is  kept  same  as  in  the 
clustering  experiment,  i.e.,  1000  mobile  nodes.  Each 
ordinary  node  has  a  small  802.11  wireless  radio  with  power 
range  175m  and  channel  bandwidth  2Mbps.  The  BNs  have 
two  802.11  radios,  one  small  radio  same  as  the  ordinary 
nodes  and  one  powerful  radio  with  power  range  800m  and 
channel  bandwidth  5Mbps.  The  mobility  model  is  “group 
mobility”  as  presented  in  [19].  30  CBR  pairs  on  top  of  UDP 
are  used  to  generate  the  traffics.  The  scope  of  backbone 
election  is  set  to  2-hop.  We  increase  the  node  mobility  from 
Om/sec  to  lOm/sec  to  compare  the  performance.  Results  are 
shown  in  Figure  8  and  Figure  9. 


Figure  8  -  Comparison  of  delivery  fraction  in  mobility 


Figure  6  -  Average  lifetime  of  cluster  head  of  multihop 
clustering 
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Figure  9  -  Comparison  of  end-to-end  delay  in  mobility 

In  Figure  8  and  Figure  9,  LANMAR/MBN  outperforms 
“flat”  LANMAR  and  AODV,  especially  when  nodes  move. 
This  is  because  it  utilizes  backbone  links  to  reduce  the 
number  of  hops  from  sources  to  destinations.  With  mobility, 
the  average  end-to-end  delay  of  AODV  is  greatly  increased. 
This  is  due  to  the  on-demand  feature  of  AODV.  For 
increasing  speed,  links  break  and  path  expire  more 
frequently.  AODV  then  must  delay  packets  as  it  searches  for 
new  paths  from  sources  to  destinations.  In  contrast, 
LANMAR  and  LANMAR/MBN  are  proactive,  thus  their 
delay  is  not  affected  greatly  by  the  speed.  LANMAR  in 
MBN  further  reduce  the  delay  using  backbone  links. 


7.  Related  Work 

In  [12],  routing  in  the  UAV  based  hierarchical  structure  is 
investigated.  Clustering  is  also  used  to  select  backbone 
nodes.  However,  only  1-hop  clustering  is  used.  The  routing 
scheme  is  fully  folded  onto  the  hierarchical  structure,  which 
centralizes  the  traffic  from  each  cluster  into  the 
corresponding  BN,  causing  potential  congestion  and  single- 
point-failure  problems.  In  contrast,  our  scheme  of 
LANMAR/MBN  shows  advantages  in  terms  of  reliability 
and  fault  tolerance. 

In  [9],  a  traditional  on-demand  routing  scheme  is  extended 
to  a  hierarchical  network.  The  scheme  does  provide 
reliability  and  fault  tolerance.  To  compare  it  to 
LANMAR/MBN,  let  us  recall  that  the  latter  shows  several 
advantages  over  “flat”  on-demand  routing.  These 
advantages  still  persist  in  the  hierarchical  structure.  Eor 
example,  the  on-demand  hierarchical  scheme  inherits  the 
long  delay  of  new  path  discovery,  which  tends  to  increase 
the  end-to-end  delay  of  data  packets,  especially  in  high 
mobility.  In  contrast,  the  LANMAR/MBN  is  proactive  and 
thus  avoids  such  a  drawback. 

8.  Conclusions 

In  this  paper  we  have  discussed  the  critical  issues  involved 
in  the  deployment  of  Backbone  Nodes  and  the  development 


of  a  scaleable  routing  protocol  for  the  MINUTEMAN 
architecture.  The  key  novelty  was  the  presence  of  the 
Mobile  Backbone  Network  (MBN)  that  must  be  properly 
exploited  by  routing.  We  have  proposed  a  new  stable 
clustering  scheme  to  deploy  the  BNs.  We  have  also 
proposed  a  LANMAR/MBN  routing  extension  that  operates 
efficiently  and  transparently  with  the  Backbone  network. 
Backbone  links  are  automatically  selected  by  the  routing 
scheme  if  they  can  reduce  hop  distance  to  remote 
destinations.  Eault  tolerance  and  system  reliability  are  also 
considered  and  achieved.  In  essence,  the  proposed  scheme 
combines  the  benefits  of  “flat”  LANMAR  routing  and 
physical  network  hierarchy.  Simulation  results  using 
Parsec/GloMoSim  platform  show  that  our  proposed 
schemes  can  establish  and  operate  a  MBN  effectively  and 
efficiently.  It  can  improve  the  network  performance 
significantly  and  it  is  robust  to  failures. 
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